Running Head: WISDOM OF THE CROWDS IN TSP

The phenomenon of the ‘wisdom of the crowds’ refers to the finding that the aggregate of a set of proposed solutions from a group of individuals performs better than the majority of individual solutions. We investigated this effect in the context of planar Euclidean traveling salesperson problem (TSP). The goal in TSPs is to estimate the shortest tour through a number of cities, represented as points in a two-dimensional display. We develop and apply an aggregation method that finds a single tour by combining the solutions from a group of individuals. Despite the fact that the aggregation method ignores spatial information, we demonstrate for most of the TSP problems that the aggregate solution tends to be closer to the optimal solution than the majority of individual solutions. Averaged across all of the TSP problems, we observe a strong wisdom of crowds effect where the averaged performance of the aggregation method outperforms even the best individual. Wisdom of the Crowds in TSP 3 Wisdom of the Crowds in Traveling Salesman Problems When judgments are made by a group of individuals, the judgment obtained by aggregating their judgments is often as good as, or might even be better than, the best individual in the group. This phenomenon, known as a wisdom of the crowds effect, is usually demonstrated for simple tasks such as estimating physical quantities or providing answers to multiple choice questions (see Surowiecki, 2004, for an overview). More recently, the effect has also been demonstrated within a single individual (Vul & Pashler, 2008). In this research, it is shown that averaging multiple estimates given by the same individual at different points in time can lead to a better answer than the individual estimates themselves. Once again, however, the focus of this work is on answering simple general knowledge questions, which have a single number as their correct answer. An important challenge for wisdom of the crowds research involves its application to problems that require much more complicated, multi-dimensional answers. Recently, for example, Steyvers, Miller, Hemmer and Lee (2009) found the wisdom of the crowds effect for ordering problems, such as listing chronologically the US presidents, or ranking cities according to their number of inhabitants. For these sorts of problems, it is not usually possible to take a simple mean or mode of individual answers to obtain a group answer. Instead, some sort of account of how people solve the problem, and the nature of individual differences, are both needed to develop an aggregation method. In this way, to tackle combinatorially complicated problems, wisdom of crowds modeling needs input from the theories and methods of cognitive science. In this paper, we investigate the wisdom of crowds for a classic complex problem solving task from computer science and operations research --which has also been a Wisdom of the Crowds in TSP 4 recent topic of study in cognitive science --known as the Traveling Salesperson Problem (TSP). We develop a method for combining individual human solutions to TSP problems, and then measure the performance of the aggregate solutions relative to the individual solutions. In TSPs, a number of nodes, or ‘cities’, must be visited in a closed cycle that visits each node once, with the goal of minimizing the distance covered over the total path. The TSP serves as a classic example of an NP-complete problem, where computationally scalable solution methods for guaranteed optimal solutions are not known (Applegate, Bixby, Chvátal, & Cook, 2006). As the problem size grows, optimal solution methods quickly require infeasible computational resources. Instead, in order to get close to optimal performance, various approximation algorithms are employed (e.g., Helsgaun, 2000, 2009). Despite the computational complexity present in TSPs, the evidence from studying human performance is that people are able to create solutions quickly, while still maintaining good performance, for at least some versions of the problem. In particular, for planar Euclidean TSPS (i.e., those where the cities can be represented as points in a low-dimensional space), people seem able to complete TSPs in approximately linear time over problem sizes (Dry, Lee, Vickers & Hughes, 2006; Graham, Joshi & Pizlo, 2000). This comes in contrast to computational approaches, whose solution times tend to be at least on the order O(n ln n) with problem size. The solutions generated by people seem to follow some basic principles consistently. They tend to connect cities along the convex hull and avoid making intersections in the path, heuristics that promote good performance (MacGregor & Omerod, 1996; van Rooij, Stege & Schactman, 2003; MacGregor, Chronicle & Omerod, Wisdom of the Crowds in TSP 5 2004). There is also evidence that human solvers are sensitive to proximity between cities, generally connecting cities with their nearest neighbors over others, (Vickers, Mayo, Heitmann, Lee & Hughes, 2004). TSP solutions have even been linked to the automatic perception of minimal structures and aesthetics. When people are asked to evaluate solutions to TSPs in terms of aesthetics, the solutions that are evaluated higher tend to also be those that have shorter lengths (Vickers, Lee, Dry, Hughes & McMahon, 2006). Earlier research by Vickers, Butavicius, Lee, and Medvedev (2001) also found similarities between solution paths created by subjects whose given goals were to create aesthetically pleasing circuits and paths created by subjects who performed the standard TSP task. Despite these general principles often being followed, however, there is also evidence for stable and significant individual differences in human TSP performance. While early results gave conflicting accounts of the level and nature of individual differences (e.g., MacGregor & Ormerod, 1996, Vickers et al. 2001), a recent reconciliation seems to have been reached, which argues for the presence of individual differences at least for sufficiently complicated problems (Chronicle, Macgregor, Lee, Ormerod & Hughes 2008). The prospect of individual differences in human TSP solutions makes it a potentially fruitful application for the wisdom of crowds idea. In particular, it raises the question of whether it is possible to combine individual solutions to find a group solution that is closer to optimal than all, or the majority, of the individual solutions. In this paper, we use previously collected data to test the wisdom of crowds idea for TSPs. In these data, each individual independently generated a solution to a given Wisdom of the Crowds in TSP 6 TSP. We develop and apply a method for aggregating these individual solutions in order to create a single aggregate tour that captures the commonalities of the individual tours. We propose an aggregation process that is restricted in two important ways. First, we assume that the cost function to evaluate the quality of a solution is not available until after the final aggregate solution is proposed. Therefore, in this situation, it is not possible to refine the solution iteratively during the aggregation process in order to optimize the tour distance. This restriction is important because, otherwise, it would be possible to ignore the human solutions altogether and just directly optimize the tours using computational means. The goal here is to see what information is collectively contained in the human solution, and the absence of the cost function during aggregation ensures that the human solutions are the only available source of information. Second, we assume that the aggregator does not have access to any spatial information, such as the location of cities. The only information available is the order in which the cities are visited on the tours proposed by a group of individuals. This restriction allows us to propose relatively simple aggregation procedures that analyze which cities tend to be connected by individuals, regardless of their spatial layout. If we can demonstrate good performance in spite of this restriction, we can argue that important information is contained in the order the cities are visited. By focusing on order information, we can develop more domain general aggregation techniques that might generalize better to other complex optimization problems that do not rely on a spatial configuration of points.